Knowledge-based automatic image segmentation

ABSTRACT

A method for medical image segmentation includes accessing and updating a knowledge-base. First, a medical image is received and a sparse landmark signature is computed based on the medical image. Next, either a representative or a cohort average reference image set is selected. A portion of either representative reference image set or the cohort average reference image set is deformed to generate mappings to the medical image set. A segmentation for each structure of interest of the medical image set is determined. The knowledge-base is searched for representative matches to form a plurality of sub-volume base sets comprising a plurality of reference image set sub-volumes. A portion of the plurality of reference image set sub-volumes of the plurality of sub-volume base sets is deformed to generate mappings from the plurality of sub-volume base sets to corresponding structures of interest of the medical image set. A weighted-average segmentation for the plurality of structures of interest in the medical image set is calculated.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation in part of and claims priority toU.S. application Ser. No. 12/845,358, filed Jul. 28, 2010.

TECHNICAL FIELD

The present disclosure relates generally to the field of automatic imagesegmentation and more specifically to the field of medical imageautomatic segmentation configured for medical applications.

BACKGROUND

When medical imaging is necessary to observe an internal organ or a setof internal organs, there are several systems that may be utilized:X-ray, magnetic resonance imaging (MRI), computed tomography (CT), andothers. When CT or MRI imagery, for example, is used, a series oftwo-dimensional images are taken from a three-dimensional volume. Here,each two-dimensional image is an image of a cross-sectional “slice” ofthe three-dimensional volume. The resulting collection oftwo-dimensional cross-sectional slices can be combined to create athree-dimensional image or reconstruction of the patient's anatomy. Thisresulting three-dimensional image or three-dimensional reconstructionwill contain the desired internal organ. This portion of thethree-dimensional image or reconstruction that contains the structure ofinterest may be referred to as a volume of interest. Note that when itis desired to observe multiple internal organs, there will then be aplurality of structures of interest as well.

These one or more structures of interest may be viewed in several ways.A first and simplest way to view the structure(s) of interest would beto merely view the original CT or MRI image slices for the patient, witheach slice containing a view of the structure(s) of interest. A second,and more complicated method to view the structure(s) of interest wouldbe to combine the series of two-dimensional cross-sectional slices intoa single three-dimensional representation where the structure(s) ofinterest may be represented as solid, opaque, or translucent, etc.,objects that may then be manipulated (e.g., rotated) to allow viewingfrom multiple angles.

One purpose of the three-dimensional reconstruction of the structure(s)of interest containing diseased or abnormal tissues or organs is thepreparation of a three-dimensional radiation therapy treatment plan.Radiation therapy treatment plans are used during medical proceduresthat selectively expose precise areas of the body, such as thosecontaining cancerous tumors, to specific doses of radiation to destroythe undesirable tissues. To develop a patient-specific radiation therapytreatment plan, information is extracted from the three-dimensionalmodel to determine perimeters such as organ shape, organ volume, tumorshape, tumor location in the organ, and the position or orientation ofseveral other structures of interest as they relate to the affectedorgan and any tumor.

The two-dimensional slices may be individually viewed on a computerscreen and with the use of conventional graphics programs, the contoursof organs or structures of interest can be traced out by hand. Contoursare connected line segments that define the outline of a structure ofinterest, which may be an organ, a portion of an organ, a tumor,diseased tissue, or a whole patient outline. Alternatively, thesestructures of interest in specific organs such as the brain or prostate,for example, may be identified with various structure-specific automaticcontouring and/or automatic segmentation software programs (subdividingan image into discrete regions) that outline or fill the shape of thestructure of interest on each two-dimensional slice of a set of slices.

SUMMARY OF THE INVENTION

This present invention provides a solution to the challenges inherent inmedical image automatic segmentation. In a method according to oneembodiment, a series of steps provide knowledge-based medical imageautomatic segmentation. After receiving a medical image, aknowledge-base is searched for a representative reference image set. Aportion of the representative reference image set is deformed togenerate mappings from the representative reference image set to themedical image. A segmentation for each organ of interest is calculated.In one embodiment, the segmentation for each organ of interest is usedas a seed for an automated structure-specific segmentation algorithm. Inanother embodiment, after the automatic segmentation algorithms arecomplete, reviewed and corrected as necessary, the medical image and itsassociated meta-data, sparse landmark signatures and structures ofinterest are added to the knowledge-base.

In a method according to another embodiment, after receiving a medicalimage, the knowledge-base is searched for a cohort average referenceimage set, wherein the knowledge-base includes at least one cohortaverage reference image set. A portion of the cohort average referenceimage set is deformed to generate mappings from the cohort averagereference image set to the medical image set. Finally, a segmentationfor each structure of interest in the medical image set is determined.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be better understood from a reading of thefollowing detailed description, taken in conjunction with theaccompanying drawing figures in which like reference charactersdesignate like elements and in which:

FIG. 1 is a simplified block diagram illustrating a system for providingknowledge-based medical image automatic segmentation, in accordance withan embodiment of the present invention;

FIG. 2 illustrates steps of an example process for knowledge-basedmedical image automatic segmentation, in accordance with an embodimentof the present invention;

FIG. 3 illustrates steps of an example process for knowledge-basedmedical image automatic segmentation, in accordance with an embodimentof the present invention;

FIG. 4 is a simplified block diagram illustrating images sets for aknowledge-based medical image automatic segmentation, in accordance withan embodiment of the present invention;

FIG. 5 illustrates steps of an example process for knowledge-basedmedical image automatic segmentation of a cohort of reference image setsin the knowledge-base, in accordance with an embodiment of the presentinvention;

FIG. 6 is a simplified block diagram illustrating knowledge-basedmedical image automatic segmentation in accordance with an embodiment ofthe present invention;

FIG. 7 is a simplified block diagram illustrating knowledge-basedmedical image automatic segmentation in accordance with an embodiment ofthe present invention;

FIG. 8 illustrates steps of an example process for knowledge-basedmedical image automatic segmentation in accordance with an embodiment ofthe present invention; and

FIG. 9 illustrates steps of an example process for knowledge-basedmedical image automatic segmentation in accordance with an embodiment ofthe present invention.

DETAILED DESCRIPTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings. While the invention will be described in conjunction with thepreferred embodiments, it will be understood that they are not intendedto limit the invention to these embodiments. On the contrary, theinvention is intended to cover alternatives, modifications andequivalents, which may be included within the spirit and scope of theinvention as defined by the appended claims. Furthermore, in thefollowing detailed description of embodiments of the present invention,numerous specific details are set forth in order to provide a thoroughunderstanding of the present invention. However, it will be recognizedby one of ordinary skill in the art that the present invention may bepracticed without these specific details. In other instances, well-knownmethods, procedures, components, and circuits have not been described indetail so as not to unnecessarily obscure aspects of the embodiments ofthe present invention.

Notation and Nomenclature

Some portions of the detailed descriptions, which follow, are presentedin terms of procedures, steps, logic blocks, processing, and othersymbolic representations of operations on data bits within a computermemory. These descriptions and representations are the means used bythose skilled in the data processing arts to most effectively convey thesubstance of their work to others skilled in the art. A procedure,computer executed step, logic block, process, etc., is here, andgenerally, conceived to be a self-consistent sequence of steps orinstructions leading to a desired result. The steps are those requiringphysical manipulations of physical quantities. Usually, though notnecessarily, these quantities take the form of electrical or magneticsignals capable of being stored, transferred, combined, compared, andotherwise manipulated in a computer system. It has proven convenient attimes, principally for reasons of common usage, to refer to thesesignals as bits, values, elements, symbols, characters, terms, numbers,or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the followingdiscussions, it is appreciated that throughout the present invention,discussions utilizing terms such as “ processing” or “accessing” or “executing” or “ storing” or “rendering” or the like, refer to the actionand processes of a computer system, or similar electronic computingdevice, that manipulates and transforms data represented as physical(electronic) quantities within the computer system's registers andmemories and other computer readable media into other data similarlyrepresented as physical quantities within the computer system memoriesor registers or other such information storage, transmission or displaydevices. When a component appears in several embodiments, the use of thesame reference numeral signifies that the component is the samecomponent as illustrated in the original embodiment.

This present invention provides a solution to the increasing challengesinherent in medical image automatic segmentation. In particular, variousembodiments of the present disclosure provide automatic segmentation ofa patient's two-dimensional medical images through the use of aknowledge base comprising a plurality of reference image sets, whereineach reference image set comprises a plurality of two-dimensionalslices. Each slice in a reference image set has been segmented andcontoured, with each structure of interest (e.g., organs, tumors,diseased tissues, etc.) labeled. As discussed in detail below, aplurality of reference image sets with similarities (e.g., similarsparse landmark signatures) to the medical image set are selected toform a base set and each deformed with a deformable image registrationalgorithm to generate mappings from the selected reference images to themedical image. Segmented structures of interest in each selectedreference image are also deformed to the medical image to generatemappings to the medical image. A weighted average segmentation for eachorgan of interest is calculated, wherein the weighted averagesegmentation for each organ of interest is used as a “seed” forstructure-specific automatic segmentation algorithms. After theautomatic segmentation algorithms for each structure of interest arecomplete, the results are reviewed and contours are corrected asrequired. After approving the medical image, the medical image and itsassociated meta-data, sparse landmark signatures and structures ofinterest are added to the knowledge-base.

Medical Image Segmentation

As is well known in the art, image segmentation is a process forreviewing a digitized medical image, and based on the merits of themedical image, differentiating or identifying structures of interest andlabeling them. For example, when reviewing a medical image of thethorax, using image segmentation, a line can be drawn around a sectionof the image and labeled as “lung.” In other words, everything withinthe line is deemed lung, and everything not inside the line is deemednot lung. Image segmentation is utilized in radiation therapies, where apractitioner needs to precisely define where the radiation treatment isto be placed and just as importantly, where the radiation treatment isnot to be placed. A structure of interest may be defined as: an organ; atumor; diseased tissues; and bones visible in the medical image, forexample. CT and MRI scans, as mentioned above, are common medical imagemodalities. However, any two-dimensional image may be segmented where apractitioner is interested in differentiating between structures ofinterest in the image. In CT and MRI scans where a plurality of slicesis generated, segmentation is usually performed on each slice in turn.For the sake of convenience and simplicity, segmenting a CT scan, forexample, refers to segmenting each slice in turn.

Medical image segmentation is used with, for example, IMRT (intensitymodulated radiation therapy), to paint radiation, or shape the radiationin the body to conform to the tumor. Segmentation is a core benefit:when a radiation treatment may be shaped so as to conform to a tumor,critical organs can then be avoided. Rather than merely fitting orshaping the radiation therapy to fit the tumor, critical organs, such asthe brain stem, for example, can be avoided. An exemplary radiationtherapy utilizes pencil beams of radiation, placing radiation into thebody at targeted locations. However, to do that, the computer runningthe IMRT treatment needs to know where the brain stem is. Treatmentplanning software for the IMRT takes medical images (e.g., CT or MRIdata sets) and identifies all the organs or structures of interest atrisk (especially those structures that are sensitive to radiation, orthat must not be irradiated regardless of their radiation sensitivity).Further, those structures of interest that are targets for the radiationtherapy are also identified. The structures as drawn on the individualslices are combined into volumes of interest, spanning multiple slices.Given the volumes drawn and the doses prescribed for each volumeidentified, the treatment planning software will calculate the bestmethod for putting the dose in.

But as medical imaging resolution increases providing additional slices,and the number of structures that must be identified, as well as theprecision with which they must be identified increases, the treatmentplanning process takes longer. This is especially apparent when eachslice must be segmented by hand. However, even with automaticsegmentation algorithms currently available, the practitioner is stilloften required to manually drop the seeds into the image, and mustreview each slice individually. “Dropping a seed” is a process whereby apractitioner selects points within an area of a medical image andidentifies them as part of a particular structure of interest (e.g., aprostate, bladder, or femur). Therefore, an excessive amount of time canbe spent manually seeding each medical image. However, even afterseeding the automatic segmentation algorithms, the practitioner is oftenrequired to edit the segmentation and contouring results to correctthem. This editing is required as the automated segmentation resultswill common produce some segmentation and/or contouring errors requiringpractitioner editing to correct. The end result is that even withautomatic segmentation software tools, the time required is often littlebetter than manually segmenting; with the necessity to drop seeds andthen frequently spend a significant amount of time editing to correctthe autosegmentation and autocontouring software results.

Automatic segmentation is further complicated by the fact that automaticsegmentation tools are often tailored to be structure specific, suchthat they only work well on certain internal organs. For example, anautomatic segmentation tool utilizing edge detection has difficulty inautomatically individually segmenting the bladder and prostate, becausethey can't be individually differentiated. When the bladder is full andtouching the prostate, the prostate-bladder barrier can't be seen. Theend result is difficulty in determining where one ends and the other onebegins. Instead, automatic segmenting tools for the prostate are forcedto rely on modeling and looking at the curvatures of the bladder andprostate. And of course, even when using a structure-specific automaticsegmentation tool, the individual seeds must still be placed within eachstructure of interest.

Knowledge-Based Image Automatic Segmentation

FIG. 1 illustrates a knowledge-base 100 in accordance with an embodimentof the present invention. A knowledge base 100 comprises a plurality ofreference image sets 102 a-102 n. Each reference image set 102 a-102 ncomprises two-dimensional slices of a three-dimensional volume. Eachreference image set 102 a-102 n further comprises meta-data, possiblyincluding a sparse landmark signature corresponding to each referenceimage, and a plurality of segments and contours corresponding tostructures of interest seen in each individual reference image slice.

In one exemplary embodiment a knowledge-base 100 initially comprises aplurality of expert cases. Each of these expert cases contains referenceimage sets 102 a-102 n with segmentations and contours that have beenindividually hand-drawn. Each structure of interest has been labeled.

In another embodiment of the knowledge-base 100, the initial pluralityof reference image sets 102 a-102 n would share common characteristics.For example, each reference image set 102 a-102 n in the knowledge-base100, would have a common characteristic in the meta-data, such as race,sex, diagnosis, treatment site, age, and size of patient, etc. Aknowledge-base 100 used predominantly in the treatment of Asian patientswould be different from a knowledge-base 100 used predominantly in thetreatment of Europeans, for example. Similarly, a knowledge-base 100used in an exclusively female practice would only need to comprisereference data sets 102 a-102 n of female patients.

FIG. 1 further illustrates a medical image processing module 110. Themedical image processing module 110 receives a image set for a currentpatient called a medical image set 112. The medical image processingmodule 110 processes the medical image set 112 to reduce each imageslice to a sparse landmark signature. The end result is a sparselandmark signature set 114. In an embodiment, the sparse landmarksignature may be computed for the entire image set, rather than for eachindividual splice.

A sparse landmark signature based on a medical image slice will allowmuch of the information contained in a medical image set to be ignored.Rather than running the algorithms pixel by pixel across each slice of amedical image set 112, which would consume an excessive amount ofcomputational time and resources, a sparse landmark signature reducesthe individual slices to only a collection of points for furtherprocessing.

For example, a medical image set 112 of a male patient with prostatecancer would have a particular sparse landmark signature set 114. If thediagnosis is prostate cancer, the three-dimensional volume of themedical image set 112 is the middle of the body. Because the pelvicgirdle defines the location of the prostate, the individual slices arescanned from top to bottom, looking for the first instance of bone(image pixels of bright intensity). This is accomplished for both sidesof the pelvic girdle, allowing the location of the femoral heads to beidentified. Next, the point where the pelvic girdle ends is alsolocated. The prostate can be found by triangulating from those threepoints.

As further illustrated in FIG. 1, after the sparse landmark signatureset 114 of the medical image set 112 is generated, reference image sets102 a-102 n can be selected based upon their similarities to the presentpatient medical image set 112 and the corresponding sparse landmarksignature set 114. In an exemplary embodiment of the present invention,a plurality of reference image sets 102 a-102 n is selected. Exemplaryembodiments can select from as few as one reference image sets 102 a-102n to as many as ten or more reference image sets 102 a-102 n. In oneexemplary embodiment, three reference image sets 102 a-102 c areselected. The selected reference image sets 102 a-102 c become a baseset 116.

In an exemplary embodiment of the present invention, the medical imageprocessing module 110 begins the reference image set 102 a-102 nselection process by reviewing meta-data attached to the current medicalimage set 112. As is well known in the art, meta-data may includeinformation associated with a particular patient at a very high levelthat is not especially personalized. It would include such details as:age, sex, diagnosis, weight, whether or not there is nodal involvement,any co-morbidities, treatment site, side of the body, and approximatelocation of tumor. The available meta-data attached to the currentmedical image set 112 is used to start a search through theknowledge-base, reviewing the meta-data attached to reference image sets102 a-102 n for corresponding matches. Through the use of meta-data, theknowledge-base can be sorted so that in an exemplary medical image setwith meta-data identifying a male patient with advanced lung cancer,only reference image sets of males with lung tumors on the same side ofthe body are reviewed. From the knowledge-base, the top reference imageset 102 a-102 c matches are selected based upon their similarities tothe present medical image set 112 meta-data and sparse landmarksignature.

A base set 116 contains as noted above in exemplary embodiments,anywhere from one to ten or more reference image sets 102 a-102 n. Aplurality of reference image sets 102 a-102 n are selected to reduce theerror inherent in each individual reference image set 102 a-102 n. It isobvious that a single reference image set 102 a-102 n won't be a perfectmatch; there will be some disagreement between the reference image set102 a-102 n and the medical image set 112. A plurality of referenceimage sets 102 a-102 n in the base set 116 allows the formation of astatistical model to serve as a reference image.

In an exemplary embodiment of the present invention, each of theselected reference images sets 102 a-102 c in the base set (in thisexemplary embodiment, three reference cases are selected) areindividually deformed to the medical image set. In other words, thepoints in the sparse landmark signature of each reference image set 102a-102 c (one slice at a time) are deformed to the points in the sparselandmark signature set 114 of the medical image set 112. In oneexemplary embodiment, a deformable image registration algorithm willtake the selected reference image sets 102 a-102 c and the medical imageset 112 which has no segmentation or contouring and morph each of thereference image sets 102 a-102 c one at a time, and one slice at a time,to look like the medical image set 112. The registration algorithm willalso transfer the contours from each reference image set 102 a-102 c tothe medical image set 112 as well. The contours are each transferredalong with the labels. By deforming each of the three selected referenceimage sets 102 a-102 c to the medical image set 112, the contours thatwere tagged to a pixel will also move as the pixels move. Each of thepoints from each of the reference image sets 102 a-102 c are deformed tothe points of the medical image set 112. In another embodiment of thepresent invention, a portion of the selected reference image sets 102a-102 n in the base set are deformed to the medical image set. Thisportion can be one or more of the reference image sets 102 a-102 n.

As multiple reference image sets 102 a-102 c are individually deformedto the medical image set 112, exemplary embodiments of the presentinvention are therefore using a statistical-based algorithm, instead ofonly an image-based algorithm. By taking the results coming from thedeformation of the three reference image sets 102 a-102 n to the medicalimage set 112 and averaging them, statistical processing will provide afinal result that is an average of the three (or whatever number ischosen) reference images 102 a-102 c. It's more statistically likelythat the average for the three top references will converge on a morecorrect answer than that any one of them will be individually correct.As illustrated in FIG. 1, the medical image processing module willtherefore deform the base set 116 to the sparse landmark signature 114of the medical image set 112 to create a statistical model 118containing segmentation and contouring of the structures of interest inthe medical image set 112.

In an exemplary embodiment, weighted-average segmentation is therebyprovided to create the statistical model 118. If all three referenceimage sets 102 a-102 c agree very well, then a simple average of thethree will be sufficient. But if two of them agree closely and the thirdhad an outlier (e.g., a contour or segmentation result for a particularstructure of interest is different from the other results), the thirdresult's outlier could be de-emphasized in the statistical model 118,making the assumption that the outlier is incorrect. Weighted-averagesegmentation for example, would look at the standard deviation of theresults for each of the structures of interest that are identified inthe statistical model 118. A simple method of weighting the results whenan outlier appears would be to discard the outlier result. It should benoted that the statistical weighted-averaging is not applied to thepatient, i.e., medical image set 112, but to each of the resultingstructures of interest within the medical image set 112. In other words,in an exemplary embodiment there can be as many weighted mixes betweenthe three reference image sets 102 a-102 c as there are structures ofinterest. The statistical model 118 therefore provides a good startingpoint for further segmentation processing. The statistical model canwith a certain amount of statistical certainty, identify what isprostate or lung for example, even if each of the individual outlinecontours aren't perfect. That is, at least what is inside the contourhas been identified as prostate or lung, as the case may be.

In an exemplary embodiment, when the weighted-average segmentationprovided to the statistical model 118 is not approved by thepractitioner, additional reference image sets that likewise containcommon meta-data characteristics and similar sparse landmark signatureswill be selected and added to the base set 116 to be registered (i.e.,deformed) to the medical image set 112. In an exemplary embodiment, theadditional reference image sets are added one at a time. Again, asdiscussed above, the sparse landmark signature of the newly addedreference image set would also be deformed to the sparse landmarksignature of the medical image set as the original three reference imagesets were. After statistical processing, an updated weighted-averagesegmentation is provided to the statistical model 118 for review by thepractitioner. Additional reference image sets can be added to the baseset 116 as required, as long as additional candidate reference imagesets remain in the knowledge-base 100, and their addition to the baseset 116 continues to improve the weighted-average segmentation of thestatistical model 118.

Further, statistical parameters of the statistical model 118 may also beevaluated, by looking at non-image based components from the referenceimage sets (shape, volume, smoothness, density, etc.) and thereby refinethe resulting statistical model 118 and updated statistical model 120.

Knowledge-Based Image Sub-Volume Segmentation

In an exemplary embodiment, the process described above for a medicalimage set 112 and a base set 116 comprising a plurality of referenceimage sets 102 a-102 n is repeated for a plurality of structures ofinterest or “sub-volumes” of the medical image set from a plurality ofpatient cases in the knowledgebase, as illustrated in FIG. 4. Asub-volume will typically contain only a single structure of interestrather than all of the structures of interest found in the medical imageset. A plurality of reference image set sub-volumes 402 a-402 n, groupedas sub-volume base sets 416 a-416 n, for each identified structure ofinterest 412 a-412 n of the medical image set 112 is selected. In otherwords, a plurality of reference image set sub-volumes 402 a-402 n areselected for each sub-volume (i.e., structure of interest) of themedical image set. For example, for a medical image set sub-volume forthe prostate, a plurality of prostate sub-volumes from reference imagesets will be selected. In other words, rather than entire referenceimage sets 102 a-102 n, only a prostate sub-volume 412 a-412 n of thereference image sets 102 a-102 n would be selected. Again, in anexemplary embodiment, the number is three. Each plurality of referenceimage set sub-volumes is therefore a sub-volume base set for aparticular structure of interest. There can be a sub-volume base set 416for each structure of interest 412 in the medical image set 112.

As described above for reference image sets 102 a-102 n and the medicalimage set 112, in the same manner, each reference image set sub-volume402 is deformed to a corresponding structure of interest 412 of themedical image set 112, just as each reference image set 102 is deformedto the medical image set 112. Further, as described above, in anotherembodiment of the present invention, a portion of the selected referenceimage sets sub-volumes 402 a-402 n in the plurality of sub-volume basesets 416 are deformed to the structures of interest 412 a-412 n in themedical image set 112. This portion can again be one or more of thereference image set sub-volumes 402 a-402 n. After a weighted-averagesegmentation for each structure of interest in the medical image set 112is completed as described above, each structure of interest orsub-volume 412 a-412 n would also have a corresponding sparse landmarksignature set 414 that is also a portion of the sparse landmarksignature set 114 of the medical image set 112. By selecting particularstructures of interest 402 (i.e., the best candidates) of referenceimage sets 102 (and their corresponding sparse landmark signatures 414for the structures of interest) to deform to the corresponding structureof interest 412 in the medical image set 112 (and its correspondingsparse landmark signature 414), the resulting weighted-averagesegmentation of each structure of interest 412 a-412 n used to createthe statistical model 118 can be further improved. This is becauserather than seeking for a reference image set 102 candidate that is agood all-around match to the patient anatomy, only the individualsub-volumes 402 or structures of interest are considered. Therefore, areference image set 102 that would not be a good candidate for allstructures of interest can still be a good sub-volume candidate.

Continuous Knowledge-Base Improvement

However, to get better results with the knowledge-base 100, additionalreference image sets need to be added to the knowledge-base 100. Inother words, additional clinical cases with their patient specificmeta-data, medical image sets, sparse landmark signatures, statisticalmodels can be added to the knowledge base after the practitioner hascompleted the patient-specific treatment plan and approved thesegmentation and contouring of the structures of interest in the medicalimage set. After the practitioner has reviewed, edited (as needed), andapproved the segmentation results, the subject patient's information canbecome a new reference image set with its own attached meta-data andtreatment information to be placed in the knowledge-base to increase thenumber of candidate reference image sets available. This new referenceimage set will then be available to be reused as part of a base set thenext time there is a patient with similar meta-data, etc. Even with verypersonalized, patient-specific medicine, these described processes allowthe creation of a closed loop: each iteration of the process allows theaddition of a new reference image set 102 to the knowledge-base.Therefore, as more reference image sets 102 are added to theknowledge-base 100, the more robust the knowledge-base 100 becomes, withimproved statistical modeling and better end results. As more referenceimage sets 102 are placed into the knowledge-base 100, the searches canbe further refined and based on additional criteria to ensure that theselected reference image sets 102 a-102 n are close matches to themedical image set 112.

Statistical Model Used as a Seed

In an exemplary embodiment, the statistical model 118 with itsweighted-average segmentation of each structure of interest can be usedas a seed itself. Rather than traditional structure-specificsegmentation tools requiring manually selected seeds before theprocessing begins, the seeds can be provided by the identifiedstructures of interest from the statistical model 118. By providingseeds that are statistically probable to be drawn in pixels inside thestructure of interest, it allows any of the previously discussedsegmentation methods to begin with improved seed selections. Further,using the statistical model's weighted-average segmentation ofstructures of interest allows these other segmentation methods to avoidhaving to start from scratch.

While there is no guarantee that a structure of interest will besegmented correctly, there is a good statistical possibility that acluster of pixels in the middle of the segmentation of the structure ofinterest will be within that actual structure of interest. In anexemplary embodiment, this central cluster of statistically significantpixels is then used as the “seed” for other automatic segmentationalgorithms.

In an exemplary embodiment, as illustrated in FIG. 1, and described inmore detail below, the statistical model 118 is used as a seed fordeformable image registration and to drive the autosegmentationcomponent of structure-specific automatic segmentation tools 130 a-130n. In an exemplary embodiment, the automatic segmentation tools 130a-130 n can be either structure-specific segmentation tools ornon-structure-specific segmentation tools. When the segmentation resultsare returned by the automatic segmentation tools 130 a-130 n to themedical image processing module 110 the results can be used to create anupdated statistical model 120 that comprises weighted-averagesegmentation of each structure of interest in the medical image set 112,as discussed in detail below.

The automatic segmentation tools 130 a-130 n use the seed provided bythe statistical model 118 to calculate the segmentation of one or morestructures of interest. These structure-specific segmentation algorithmscan use a structure-specific model to aid in completing the segmentationthat will look at all the data within the image that has been labeled asa seed for the structure of interest (e.g., the prostate or lung). Thisgives the structure-specific algorithm a very good starting point. Thestructure-specific algorithms work best when they are working on thespecific structures they are designed to segment.

The statistical weighted averaging can also be extended to statisticallyweighted-averaging of these different algorithms depending on theirabilities and whether there is something about the particular meta-dataor the image that would make one algorithm more accurate than another.This allows the results from a plurality of algorithms to be averaged toarrive at an even more statistically accurate definition of a structureof interest by comparing the results from a plurality ofstructure-specific algorithms. The end result would be the medical imageprocessing module calculating an updated statistical model 120 thatcomprises a weighted-average segmentation of each structure of interestin the medical image set 112. As discussed above, this updatedstatistical model 120 after being reviewed, editing, and finallyapproved by the practitioner for use with the current patient, may beadded to the knowledge-base as a new reference image set 102.

Knowledge-Based Autosegmentation Algorithm Improvements

In an exemplary embodiment, the above described processes can include anupdate to the statistical tools, by performing a difference calculation.The difference calculation compares what the system calculated to whatwas ultimately accepted by the practitioner, identifies the point(s)that failed, and acts to modify the behavior of the algorithm. Byback-projecting the difference, changes are made to the algorithmparameterization until an optimal solution results for the individualpatient case. Weighting functions can be adjusted to reach the optimalresults. The patient specific parameterizations can then be incorporatedinto a population-parameterization of each algorithm.

In other words, if the system was failing to recognize a pelvis case,for instance, the selection of sparse landmarks would have to beadjusted so that a pelvis case is properly recognized. In anotherexample, the way that deformal image registration is identifying astructure of interest may have to be changed if the location is correct,but the segmentation/contour result is incorrect. For example, if a gapexists between the computed and hand-drawn edge of an organ of interest,then the edge detection algorithm can be adjusted to correct it. Knowingwhere the gap was and where the correction was made, the correction canbe worked into the particular algorithm to improve it. If a segmentationalgorithm fails similarly throughout a population of patients, itsweighting factor for the identified failure mode would be decreased whencomputing a weighted average between different segmentation algorithms.

Knowledge-Based Autosegmentation Processes

FIG. 2 illustrates the steps of a process in accordance with anembodiment of the present invention. Where the process includes stepsthat have already been explained in detail, they will not be repeatedhere. In step 202, a new medical image set 112 is loaded. The medicalimage set 112 may be loaded into the memory of a computer system. Instep 204, landmark features (i.e., points on the medical image set 112)are extracted to create a sparse landmark signature 114.

In step 206, a knowledge-base 100 is scanned to search for referenceimage sets 102 a-102 n with similar sparse landmark signatures 114 andsimilar meta-data. In an exemplary embodiment, the sparse landmarksignatures 114 serve as an index for the knowledge-base 100. In step208, n reference image sets 102 a-102 n are selected to form a base set116. In an exemplary embodiment, n=3. In step 210, a portion of thereference image sets 102 a-102 n are registered to the medical image set112. This portion can be two or more of the selected reference imagesets 102 a-102 n. In one exemplary embodiment, all of the selectedreference image sets 102 a-102 n are registered. By deforming thereference image set to the medical image, the segmented and contouredstructures of interest in the reference image set are also applied tothe medical image set.

In step 212, a weighted average structure-specific segmentation iscompleted to form the statistical model 118. In other words, weightedaverages are calculated for each structure to reach a weighted-averagestructure segmentation. As noted above, the statistical model 118 maynow serve as a seed for use in other segmentation tools, including thosethat are structure-specific.

In step 214, the practitioner or user reviews, edits and approves thesegmentation results. In step 216, structure-specific statisticalparameters are computed and improved when possible (by looking atnon-image based components from the reference image sets (shape, volume,smoothness, density, etc.). Then in step 218, the knowledge-base isupdated with the medical image set 112 and its corresponding structureset (statistical model 118), sparse landmark signature set 114, andmeta-data as a new reference image set 102.

FIG. 3 illustrates the steps of a process in accordance with anembodiment of the present invention. Where the process includes stepsthat have already been explained in detail, they will not be repeatedhere. Step 302 begins after a statistical model 118 has been calculated.In step 302, statistical parameters describing each segmented structureof interest are further evaluated. In step 304, sub-volumes 412 a-412 nfrom the medical image set 112, wherein a sub-volume contains or is astructure of interest, are loaded into the computer for processing.These sub-volumes 412 a-412 n are the structures of interest with theirown sparse landmark signatures 414 that have already been segmented inFIG. 2, as described above. In other words, each sub-volume comprises412 a-412 n a sub-volume sparse landmark signature 414. In step 306, nreference image set sub-volumes 402 a-402 n are loaded for eachstructure of interest (i.e., sub-volume) in the medical image set 112 toform corresponding sub-volume base sets 416 a-416 n. In one embodiment,n=3. Each of the reference image set sub-volumes 402 a-402 c of theplurality of sub-volume base sets 416 a-416 n is selected for acorresponding structure of interest 412 a-412 n in the medical image. Inone embodiment, each reference image set sub-volume 402 can come from adifferent reference image set 102. In other words, if there are fifteenstructures of interest in the medical image set 112, then there will beforty-five reference image sets 102 a-102 n providing the necessaryforty-five reference image set sub-volumes 402 a-402 n contained in thefifteen sub-volume base sets 416 a-416 n. In another embodiment, asingle reference set 102 can provide a plurality of reference image setsub-volumes 402 a-402 n.

In step 308, a portion of the reference image set sub-volumes 402 a-402n and their segmented and contoured structures of interest are deformedor registered to the corresponding structure of interest 412 a-412 n ofthe medical image set 112. This portion can be two or more of theselected reference image sets 402 a-402 n. In one exemplary embodiment,all of the selected reference image set sub-volumes 402 a-402 n areregistered or deformed. With the reference image set sub-volumes 402a-402 n from each sub-volume base set 416 a-416 n deforming to thecorresponding structure of interest 412 a-412 n of the medical image set112. In step 310, a weighted-average segmentation value is calculatedfor each structure of interest 412 a-412 n in the medical image set 112.Calculating the weighted-average segmentation value for each structureof interest 412 a-412 n in the medical image set 112 allows thecalculation of an updated statistical model 118.

In step 312 each structure of interest 412 a-412 n in the statisticalmodel 118 is used as a seed and each post-processed by a plurality ofstructure-specific segmentation tools. Each structure of interestsegmentation is used as a seed for a plurality of structure-specificsegmentation tools. In step 314, a weighted-average segmentation resultis calculated for each structure of interest based upon the segmentationresults for each structure of interest 412 a-412 n provided by thestructure-specific segmentation tools. As mentioned above, thisweighted-average segmentation result for each structure of interest 412a-412 n is the updated statistical model 120.

In step 316, the practitioner or user reviews, edits and approves theweighted-average segmentation results from step 314. In step 318,structure-specific statistical parameters are computed and improved whenpossible (by looking at non-image based components from the referenceimage sets (shape, volume, smoothness, density, etc.). Then in step 320,the knowledge-base is updated with the medical image set 112 and itscorresponding structure set (statistical model 120), sparse landmarksignature 114, and meta-data as a new reference image set 102.

FIG. 5 illustrates the steps of a process in accordance with anembodiment of the present invention. In step 502, a cohort of patientsor reference image sets 102 a-102 n within the knowledge-base areselected that share common characteristics. These common characteristicscan include diagnosis, sex, age, race, treatment site, etc. In step 504,all members of the cohort are deformed into a single aggregated dataset,forming a single representative case. In step 506, geometric analysis isperformed on the radiation therapy for structures of interest and theassociated radiation therapy doses. In step 508, cohort representativesare compared to other cohort representative cases.

Individual and Cohort Average Image Automatic Segmentation

In further embodiments, as illustrated in FIGS. 6 and 8, rather thanselecting a plurality of reference image sets 102 a-102 n to form a baseset 116, a single reference image set 102 may be selected and registeredto the medical image set 112 to form a segmented image 602. In a furtherembodiment, as illustrated in FIGS. 7 and 9, rather than selecting aplurality of reference image sets 102 a-102 n that are then individuallyregistered to the medical image set 112 to form a statistical model 118based upon the weighted-average segmentation of the individuallyregistered plurality of reference image sets 102 a-102 n, a cohortaverage reference image set 702 is selected and registered to themedical image set 112 to form a segmented image 602. As described below,a cohort average reference image set 702 is a reference image set 102that is an average of all or part of the reference image sets 102 a-102n in a cohort.

As illustrated in FIG. 6, and further illustrated in FIG. 1, after thesparse landmark signature set 114 of the medical image set 112 isgenerated, a reference image set 102 can be selected based upon itssimilarities to the present patient medical image set 112 and thecorresponding sparse landmark signature set 114. In an exemplaryembodiment of the present invention, the medical image processing module110 begins the reference image set 102 selection process by reviewingmeta-data attached to the current medical image set 112. As is wellknown in the art, meta-data may include information associated with aparticular patient at a very high level that is not especiallypersonalized. It would include such details as: sex, age, diagnosis,height, weight, whether or not there is nodal involvement, anyco-morbidities, treatment site, side of the body, and approximatelocation of the tumor, to name but a few. For example, a cohort maycomprise reference image sets 102 a-102 n contained within a specifiedweight range (e.g., 160-200 pounds). As discussed above, the availablemeta-data attached to the current medical image set 112 is used to starta search through the knowledge-base, reviewing the meta-data attached toreference image sets 102 a-102 n for corresponding matches. In oneembodiment, the associated reference image sets 102 a-102 n of aselected cohort is searched. As discussed above, a selected cohort wouldcontain reference image sets 102 a-102 n with at least one commoncharacteristic. In other words, first a particular cohort would beselected, and then a reference image set 102 contained in the selectedcohort will be selected. A reference image set 102 in the selectedcohort can be selected based upon its similarities to the presentpatient medical image set 112 and the corresponding sparse landmarksignature set 114. In an additional embodiment, rather than selecting a“best match” reference image set 102, a reference image set 102 in theknowledge-base is selected randomly. In additional embodiments, arandomly selected reference image set 102 is randomly selected from thereference image sets 102 a-102 n in a selected cohort.

In an exemplary embodiment of the present invention, the selectedreference image set 102 is deformed to the medical image set. In otherwords, the points of the sparse landmark signature of the referenceimage set 102 (one slice at a time) is deformed to the points in thesparse landmark signature 112 of the medical image set 112. In oneexemplary embodiment, a deformable image registration algorithm willtake the selected reference image set 102 and the medical image set 112,which has no segmentation or contouring, and morph the reference imageset, one slice at a time, to look like the medical image set 112. Theregistration algorithm will also transfer the contours from the selectedreference image set 102 to the medical image set 112 as well. Thecontours are each transferred along with the labels. By deforming theselected reference image set 102 to the medical image set 112, thecontours that were tagged with a pixel will also move as the pixelsmove. Each of the points from the selected reference image set 102 aredeformed to the points of the medical image set 112. In anotherembodiment of the present invention, a portion of the selected referenceimage set 102 is deformed to the medical image set 112. As discussedabove, and illustrated in FIG. 1, the medical image processing modulewill therefore deform the selected reference image set 102 to the sparselandmark signature 114 of the medical image set 112 to create asegmented image 602, as illustrated in FIG. 6, containing segmentationand contouring of the structures of interest in the medical image set112.

Further, parameters of the segmented image 602 can also be evaluated, bylooking at non-image based components from the reference image set 102(shape, volume, smoothness, density, etc.) and thereby refine theresulting segmented image 602.

In the exemplary embodiment discussed above for knowledge-based imagesub-volume segmentation, the process described above for a medical imageset and a base set 116 comprising a plurality of reference image sets102 a-10 n, is performed for a plurality of structures of interest or“sub volumes” of the segmented medical image set 602 from a plurality ofpatient cases in the knowledge-base, as illustrated in FIG. 4, anddiscussed in detail above.

A process for registering a selected reference image set 102 to themedical image set 112 is illustrated in FIG. 8. In step 802, a newmedical image set 112 is loaded into the medical image processing module110. In step 804, landmark features are extracted from the medical imageset 112 to create a sparse landmark signature. In step 806, theknowledge-base is scanned for cases with similar sparse signatures andmeta-data. As discussed above, the entire knowledge-base may be scanned,or the reference image sets 102 a-102 n of a selected cohort may bescanned. In step 808, a single, “best-match” reference image set 102 isselected and loaded into the medical image processing module 110. Instep 810, the best match reference image set 102 is registered (deformedor affined) to the medical image set 112 along with structures ofinterest. In step 812, the user reviews the segmentation results, editsthe segmentation result and approves the final segmentation result. Instep 814, structure-specific statistical parameters are computed asdiscussed above. Finally, in step 816, the knowledge-base is updatedwith the medical image set 112 and the corresponding structure set.

In a further embodiment, rather than individually registering selectedreference image sets 102 a-102 n to the medical image set 112 andcalculating a weighted-average based statistical model 118 as discussedin detail above, a cohort average reference image set 702 may beselected from the knowledge base. In an embodiment of the presentinvention, each cohort contains at least one cohort average referenceimage set 702 corresponding to the reference image sets 102 a-102 n inthe cohort. In other words, each of the reference images sets 102 a-102n in a cohort are averaged to create a reference image set that is anaverage of all the reference image sets in the particular cohort. Such acohort average reference image set 702 may be an average of all of thereference image sets 102 a-102 n in a cohort, or just a selected portionof them. In a further embodiment, there can be a cohort averagereference image set 702 that is an average of all the reference imagesets 102 a-102 n in the knowledge base or of a portion of the referenceimage sets 102 a-102 n in the knowledge base. In other words, a hundredor more (e.g., 1000) reference image sets 102 a-102 n may be averagedtogether to produce a cohort average reference image set 702 that is anaverage of all the reference image sets in the knowledge base, or astatistically significant portion of them. It is appreciated that acohort average reference image set 702 for each cohort or portions ofthe knowledge-base are exemplary in nature and not intended to limit thescope of the present invention.

As illustrated in FIG. 7, and further illustrated in FIG. 1, after thesparse landmark signature set 114 of the medical image set 112 isgenerated, a cohort of reference image sets 102 a-102 n can be selectedbased upon its similarities to the present patient medical image set 112and the corresponding sparse landmark signature set 114. In other words,a cohort can be selected based upon the similarities of its cohortaverage reference image set 602 to the present patient medical image set112 and the corresponding sparse landmark signature set 114. In anexemplary embodiment of the present invention, the medical imageprocessing module 110 begins the cohort selection process by reviewingmeta-data attached to the current medical image set 112. As is wellknown in the art, meta-data may include information associated with aparticular patient at a very high level that is not especiallypersonalized. It would include such details as: sex, age, diagnosis,height, weight, whether or not there is nodal involvement, anyco-morbidities, treatment site, side of the body, and approximatelocation of the tumor, to name but a few. For example, a cohort maycomprise reference image sets 102 a-102 n contained within a specifiedweight range (e.g., 160-200 pounds). As discussed above, the availablemeta-data attached to the current medical image set 112 is used to starta search through the knowledge-base, reviewing the meta-data attached tocohorts of reference image sets 102 a-102 n for corresponding matches.As discussed above, a selected cohort would contain reference image sets102 a-102 n with at least one common characteristic. Once a cohort isselected, the cohort average reference image set 702 of the selectedcohort is loaded into the medical image processing module 110. Asdiscussed above, rather than selecting a cohort average reference imageset 702 corresponding to a particular cohort, a cohort average referenceimage set 702 may be selected that is an average of all or only aportion of the reference image sets 102 a-102 n in the knowledge base.

In an exemplary embodiment of the present invention, the selected cohortaverage reference image set 702 is deformed to the medical image set. Inother words, the points of the sparse landmark signature of the cohortaverage reference image set 702 (one slice at a time) are deformed tothe points in the sparse landmark signature 112 of the medical image set112. In one exemplary embodiment, a deformable image registrationalgorithm will take the selected cohort average reference image set 702and the medical image set 112, which has no segmentation or contouring,and morph the reference image set, one slice at a time, to look like themedical image set 112. The registration algorithm will also transfer thecontours from the selected cohort average reference image set 702 to themedical image set 112 as well. The contours are each transferred alongwith the labels. By deforming the selected cohort average referenceimage set 702 to the medical image set 112, the contours that weretagged with a pixel will also move as the pixels move. Each of thepoints from the selected cohort average reference image set 702 aredeformed to the points of the medical image set 112.

In another embodiment of the present invention, a portion of theselected cohort average reference image set 702 is deformed to themedical image set 112. As discussed above, and illustrated in FIG. 1,the medical image processing module will therefore deform the selectedcohort average reference image set 702 to the sparse landmark signature114 of the medical image set 112 to create a segmented image 602, asillustrated in FIG. 7, containing segmentation and contouring of thestructures of interest in the medical image set 112. In anotherembodiment, the statistical average of all the reference image sets in acohort of reference image sets within the knowledge-base are determinedby determining a statistical average of a quantity of deformably oraffinely registered reference image sets in the cohort, wherein thequantity of reference image sets is one of a portion of the referenceimage sets in the cohort and all the reference image sets in the cohort,and determining a statistical average of a quantity of deformably oraffinely registered reference image sets in the knowledge-base, whereinthe quantity of reference image sets is one of a portion of thereference image sets in the knowledge-base and all the reference imagesets in the knowledge-base.

As discussed above, parameters of the segmented image 602 can also beevaluated, by looking at non-image based components from the cohortaverage reference image set 702 (shape, volume, smoothness, density,etc.) and thereby refine the resulting segmented image 602.

In the exemplary embodiment discussed above for knowledge-based imagesub-volume segmentation, the process described above for a medical imageset and a base set 116 comprising a plurality of reference image sets102 a-10 n, is performed for a plurality of structures of interest or“sub volumes” of the segmented medical image set 602 from a plurality ofpatient cases in the knowledge-base, as illustrated in FIG. 4, anddiscussed in detail above.

A process for registering a selected cohort average reference image set702 to the medical image set 112 is illustrated in FIG. 9. In step 902,a new medical image set 112 is loaded into the medical image processingmodule 110. In step 904, landmark features are extracted from themedical image set 112 to create a sparse landmark signature. In step906, the knowledge-base is scanned for a cohort with similar sparsesignatures and meta-data. In step 908, a cohort average reference imageset 702 from the selected cohort is loaded into the medical imageprocessing module 110. In step 910, the cohort average reference imageset 702 is registered (deformed) to the medical image set 112 along withstructures of interest. In step 912, the user reviews the segmentationresults, edits the segmentation result and approves the finalsegmentation result. In step 914, structure-specific statisticalparameters are computed as discussed above. Finally, in step 916, theknowledge-base is updated with the medical image set 112 and thecorresponding structure set.

Although certain preferred embodiments and methods have been disclosedherein, it will be apparent from the foregoing disclosure to thoseskilled in the art that variations and modifications of such embodimentsand methods may be made without departing from the spirit and scope ofthe invention. It is intended that the invention shall be limited onlyto the extent required by the appended claims and the rules andprinciples of applicable law.

1. A method for segmenting image data sets, comprising: receiving amedical image set; selecting a representative reference image set from aknowledge-base; deforming a portion of the representative referenceimage set to generate mappings from the representative reference imageset to the medical image set; and determining a segmentation for aplurality of structures of interest in the medical image set based uponmappings to the medical image set.
 2. The method of claim 1, furthercomprising: using the segmentation for the plurality of structures ofinterest as a seed for at least one structure-specific segmentationalgorithm.
 3. The method of claim 1, wherein the representativereference image set is selected from a cohort of reference image setswithin the knowledge-base that share common characteristics, wherein thecommon characteristics comprise at least one of: weight, sex, diagnosis,stage, responder versus non-responders, age, treatment site, and size.4. The method of claim 1, further comprising: computing a sparselandmark signature based on the medical image set, wherein the sparselandmark signature comprises a plurality of points.
 5. The method ofclaim 1, further comprising: updating a plurality of statisticalanalysis tools that determine the segmentation.
 6. The method of claim1, wherein deforming a portion of the representative reference imageset, comprises: deforming the portion of the representative referenceimage set with a deformable image registration algorithm.
 7. The methodof claim 1, wherein deforming a portion of the representative referenceimage set, comprises: deforming segmented structures of interest fromthe portion of the representative reference image set to create a map tothe medical image set.
 8. The method of claim 4, wherein deforming aportion of the representative reference image set, comprises: deformingpoints of a sparse landmark signature of the portion of therepresentative reference image set to the points of the sparse landmarksignature of the medical image set.
 9. The method of claim 4, whereinselecting the representative reference image set from theknowledge-base, comprises: comparing meta-data from the medical imageset to meta-data of the reference image sets in the knowledge-base, andcomparing sparse landmark signatures of the medical image set to sparselandmark signatures of the reference image sets in the knowledge-base,wherein the selected representative reference image set is a best matchreference image set.
 10. The method of claim 1, wherein selecting therepresentative reference image set from the knowledge-base, comprisesone of: randomly selecting a reference image set from a cohort ofreference image sets in the knowledge-base, wherein the knowledge-basecontains at least one cohort; and randomly selecting a reference imageset from the knowledge-base.
 11. The method of claim 1, furthercomprising: reviewing and correcting the contours; and adding theapproved medical image set and corresponding structures of interest tothe knowledge-base as a new reference image set.
 12. A method forsegmenting image data sets, comprising: receiving a medical image set;selecting a cohort average reference image set from a knowledge-base;deforming a portion of the cohort average reference image set togenerate mappings from the cohort average reference image set to themedical image set; and determining a segmentation for a plurality ofstructures of interest in the medical image set based upon mappings tothe medical image set.
 13. The method of claim 12, further comprising:using the segmentation for the plurality of structures of interest as aseed for at least one structure-specific segmentation algorithm.
 14. Themethod of claim 12, wherein the cohort average reference image set is astatistical average of the registered reference image sets in a cohortof reference image sets within the knowledge-base that share commoncharacteristics, wherein the common characteristics comprise at leastone of: weight, sex, diagnosis, stage, responder versus non-responders,age, treatment site, and size.
 15. The method of claim 12, furthercomprising: computing a sparse landmark signature based on the medicalimage set, wherein the sparse landmark signature comprises a pluralityof points.
 16. The method of claim 12, further comprising: updating aplurality of statistical analysis tools that determine the segmentation.17. The method of claim 12, wherein deforming a portion of the cohortaverage reference image set, comprises: deforming the portion of thecohort average reference image set with a deformable image registrationalgorithm.
 18. The method of claim 12, wherein deforming a portion ofthe cohort average reference image set, comprises: deforming segmentedstructures of interest from the portion of the cohort average referenceimage set to create a map to the medical image set.
 19. The method ofclaim 15, wherein deforming a portion of the cohort average referenceimage set, comprises: deforming points of a sparse landmark signature ofthe portion of the cohort average reference image set to the points ofthe sparse landmark signature of the medical image set.
 20. The methodof claim 15, wherein selecting the cohort average reference image setfrom the knowledge-base, comprises: comparing meta-data from the medicalimage set to meta-data of the cohorts in the knowledge-base, andcomparing sparse landmark signatures of the medical image set to sparselandmark signatures of the cohort average reference image sets in theknowledge-base, wherein the selected cohort average reference image setis a best match cohort average reference image set.
 21. The method ofclaim 14, wherein determining the statistical average of all thereference image sets in a cohort of reference image sets within theknowledge-base, comprises one of: determining a statistical average of aquantity of deformably or affinely registered reference image sets inthe cohort, wherein the quantity of reference image sets is one of aportion of the reference image sets in the cohort and all the referenceimage sets in the cohort; and determining a statistical average of aquantity of deformably or affinely registered reference image sets inthe knowledge-base, wherein the quantity of reference image sets is oneof a portion of the reference image sets in the knowledge-base and allthe reference image sets in the knowledge-base.
 22. The method of claim12, further comprising: reviewing and correcting the contours; andadding the approved medical image set and corresponding structures ofinterest to the knowledge-base as a new reference image set.
 23. Amethod for segmenting image data sets, comprising: receiving a medicalimage set; selecting one of: a representative reference image set from aknowledge-base; and a cohort average reference image set from theknowledge-base; deforming a portion of the selected one of therepresentative reference image set and the cohort average referenceimage set to generate mappings from the selected one of therepresentative reference image set and the cohort average referenceimage set to the medical image set; determining a segmentation for aplurality of structures of interest in the medical image set based uponmappings to the medical image set, searching the knowledge-base forrepresentative matches to form a plurality of sub-volume base sets,wherein a sub-volume base set comprises a plurality of reference imageset sub-volumes; deforming a portion of the plurality of reference imageset sub-volumes of the plurality of sub-volume base sets to generatemappings from the plurality of sub-volume base sets to correspondingstructures of interest of the medical image set; and determining aweighted-average segmentation for the plurality of structures ofinterest in the medical image set based upon mappings to the pluralityof structures of interest in the medical image set.
 24. The method ofclaim 23, further comprising: using the weighted-average segmentationfor the plurality of structures of interest as a seed for at least onestructure-specific segmentation algorithm.
 25. The method of claim 23,wherein the representative reference image set is selected from a cohortof reference image sets within the knowledge-base that share commoncharacteristics, wherein the common characteristics comprise at leastone of: weight, sex, diagnosis, stage, responder versus non-responders,age, treatment site, and size.
 26. The method of claim 23, wherein thecohort average reference image set is a statistical average of all thereference image sets in a cohort of reference image sets within theknowledge-base that share common characteristics, wherein the commoncharacteristics comprise at least one of: weight, sex, diagnosis, stage,responder versus non-responders, age, treatment site, and size, andwherein the knowledge-base includes at least one cohort averagereference image set.
 27. The method of claim 23, further comprising:computing a sparse landmark signature based on the medical image set,wherein the sparse landmark signature comprises a plurality of points,and wherein the sparse landmark signature for the medical image setfurther comprises a plurality of sparse landmark signatures forcorresponding structures of interest in the medical image set.
 28. Themethod of claim 23, further comprising: updating a plurality ofstatistical analysis tools that determine the weighted-averagesegmentation.
 29. The method of claim 23, wherein deforming a portion ofthe selected one of the representative reference image set and thecohort average reference image set and deforming a portion of theplurality of reference image set sub-volumes of the plurality ofsub-volume base sets, comprises: deforming the portion of the selectedone of the representative reference image set and the cohort averagereference image set with a deformable image registration algorithm; anddeforming the portion of the plurality of reference image setsub-volumes of the plurality of sub-volume base sets with the deformableimage registration algorithm.
 30. The method of claim 23, whereindeforming a portion of the selected one of the representative referenceimage set and the cohort average reference image set and deforming aportion of the plurality of reference image set sub-volumes of theplurality of sub-volume base sets, comprises: deforming segmentedstructures of interest from the portion of the selected one of therepresentative reference image set and the cohort average referenceimage set to create a map to the medical image set; and deformingsegmented structures of interest from the portion of the plurality ofreference image set sub-volumes of the plurality of sub-volume base setsto create a map to the corresponding structures of interest of themedical image set.
 31. The method of claim 26, wherein deforming aportion of the selected one of the representative reference image setand the cohort average reference image set and deforming a portion ofthe plurality of reference image set sub-volumes of the plurality ofsub-volume base sets, comprises: deforming points of a plurality ofsparse landmark signatures of the selected one of the representativereference image set and the cohort average reference image set to thepoints in the sparse landmark signature of the medical image set; anddeforming points of a plurality of sparse landmark signatures of theportion of the plurality of reference image set sub-volumes of theplurality of sub-volume base sets to the points in the correspondingsparse landmark signatures of the plurality of structures of interest ofthe medical image set.
 32. The method of claim 23, wherein selecting therepresentative reference image set from the knowledge-base, comprisesone of: randomly selecting a reference image set from a cohort ofreference image sets in the knowledge-base, wherein the knowledge-basecontains at least one cohort; randomly selecting a reference image setfrom the knowledge-base; comparing sparse landmark signatures of themedical image set to sparse landmark signatures of the reference imagesets in the knowledge-base, wherein the selected representativereference image set is a best match reference image set, and comparingmeta-data from the medical image set to meta-data of the reference imagesets in the knowledge-base.
 33. The method of claim 23, whereinselecting the cohort average reference image set from theknowledge-base, comprises: comparing meta-data from the medical imageset to meta-data of the cohorts in the knowledge-base, and comparingsparse landmark signatures of the medical image set to sparse landmarksignatures of the cohort average reference image sets in theknowledge-base, wherein the selected cohort average reference image setis a best match cohort average reference image set.
 34. The method ofclaim 26, wherein searching the knowledge-base for representativematches to form a sub-volume base set, comprises: comparing meta-datafrom the medical image set to meta-data of reference image sets in theknowledge-base, and comparing sparse landmark signatures of eachstructure of interest of the medical image set to sparse landmarksignatures of corresponding structures of interest of reference imagesets in the knowledge-base.
 35. The method of claim 23, furthercomprising: reviewing and correcting the contours; and adding theapproved medical image set and corresponding structures of interest tothe knowledge-base as a new reference image set.